A Market Protocol for Decentralized Task Allocation and Scheduling: Thesis Proposal Committee Members

I present a decentralized market protocol for allocating and scheduling tasks among agents to form supply chains. The protocol resolves constraints arising from hierarchical subtask relationships and resource scarcity. Agents negotiate through auctions according to speci ed bidding policies, and the auctions in turn determine the allocations and prices of the tasks and resources. I describe some equilibrium, convergence, and optimality properties of the system. I propose to further analyze the convergence properties, analyze adaptivity and agent strategic behavior, and develop a model of task allocation for a concrete problem domain. 1 Overview In a multiagent system (MAS), we must often address the problem of allocating resources and e ort in such a way that the resulting collection of agents can accomplish some complex tasks. In order to perform a particular task, an agent may need to achieve some subtasks, which may in turn be delegated to other agents, forming a supply chain through a hierarchy of task achievement. We may also need to schedule the use of resources around time constraints in the supply chain. This problem is complicated if the agents contend for limited resources, which may preclude the use of simple greedy allocation strategies. Furthermore, we are often constrained to allocate tasks and resources in a decentralized manner. Because the agents are autonomous, we must generally assume that they have specialized knowledge about their own capabilities but limited knowledge about other individuals and the large-scale structure of the problem. Additionally, technological or other design constraints may require us to compute the allocation of each resource individually. Decentralizing the computation can never improve the quality of an allocation; a centralized system could do at least as well by simulating the decentralized system. However, given the decentralized nature of the problem as an exogenous constraint, we seek to design principled, e ective, resource allocation protocols. Several researchers have performed in-depth analyses of task allocation without hierarchical dependencies [1, 17, 20]. Others have considered task allocation and scheduling that include variable costs, but not strict resource limitations [2, 5, 19]. I use a market-oriented programming approach to solve task allocation and scheduling problems with hierarchical dependencies and resource limitations. In the market-oriented approach, we characterize agent decisions as resource allocation problems. Market mechanisms facilitate agent negotiations and determine nal allocations. Prices, speci ed by the market mechanisms, provide information about the relative global value of the various tasks and resources, and agents use the prices to guide their local decision making. Thus we achieve complex, multilateral allocations of resources in a decentralized manner, via relatively simple bilateral negotiations and exchanges. Moreover, solution methods and analytical techniques from economics can provide useful concepts for designing and understanding market systems. Experience with the market-oriented programming approach has veri ed 1 that it works predictably and e ectively in several convex domains [15, 29], characterized by in nite divisibility of resources and nonincreasing returns to scale. However, many important resource allocation problems|such as task allocation|are inherently discrete, violating the standard general conditions for market e ectiveness. Within economics, protocols for allocation of discrete goods have been studied under the heading of auction theory. Although much of the auction literature focuses on the allocation of single items [7, 11, 26], several studies address the more challenging problem of allocating multiple items [6, 8, 10, 16, 23, 25, 31], and recent experience with the United States FCC radio spectrum auctions [12, 13] has prompted further economic interest in these problems [14]. In this proposal I present my completed work, and propose future work, on a decentralized market protocol for allocating and scheduling discrete tasks among agents in asynchronous systems under conditions of resource scarcity. The protocol builds supply chains in a bottom-up fashion using strictly local knowledge and communication. I describe my formulation of the task allocation problem in Section 2. I discuss price systems and competitive equilibrium in Section 3, and survey some market protocols for resource allocation in 4. I present and analyze the SAMP-SB protocol, a variant of simultaneous ascending auction protocols, in Section 5. I describe and analyze a protocol for detecting system quiescence in a distributed fashion in Section 6. I describe related task allocation and scheduling work in Section 7. Finally, I propose my research plan in Section 8. Appendix A provides a reference for the notation I use throughout this proposal. Appendix B provides a summary of the major concepts in this proposal. 2 The Task Allocation Problem In the task allocation problem, we are interested in the achievement of some task or tasks, which may be performed by various agents at varying times. In order to perform a particular task, an agent may need to achieve some subtasks, which may in turn be delegated to other agents, forming a supply chain through a hierarchy of task achievement. Constraints on the task assignment arise from resource contention, where agents would need a common resource (e.g., a subtask achievement, or something tangible like a piece of equipment) to accomplish their own tasks. We may also need to schedule 2 Airfield 2b Fighter Squad 2 Paratroopers 2 Bomber Squad 2 Airfield 2a Airfield 1b Airfield 1a Fighter Squad 1 Paratroopers 1 Bomber Squad 1 Commander 1 Air Attack Figure 1: A task allocation problem in the air combat domain, with no resource contention. the use of resources around time constraints in the supply chain. Tasks are performed on behalf of particular agents; if two agents need a subtask then it would have to be performed twice to satisfy them both. In this way, tasks are the same as any other discrete resource. Hence, we make no distinction in our model, and use the term \good" to refer to any task or resource provided or needed by agents. 2.1 An Example Task Allocation Problem Consider the problem of allocating air squadrons and elds in a simpli ed air combat domain. Figure 1 shows the various resources available. Commander 1 wants to conduct an air attack, requiring a bomber squad and ghter squad, each of which in turn requires the use of a local air eld. Figure 2 shows that the tasks and resources can be allocated easily, because there is no contention. 3 Airfield 2b Fighter Squad 2 Paratroopers 2 Bomber Squad 2 Airfield 2a Airfield 1b Airfield 1a Fighter Squad 1 Paratroopers 1 Bomber Squad 1 Commander 1 Air Attack Figure 2: An allocation that satis es Commander 1. 4 Commander 1 Air Attack Commander 2 Ground Attack Airfield 2b Fighter Squad 2 Paratroopers 2 Bomber Squad 2 Airfield 2a Airfield 1b Airfield 1a Fighter Squad 1 Paratroopers 1 Bomber Squad 1 Figure 3: Commanders 1 and 2 contend for limited air eld resources. 5 Commander 1 Air Attack Commander 2 Ground Attack Airfield 2b Fighter Squad 2 Paratroopers 2 Bomber Squad 2 Airfield 2a Airfield 1b Airfield 1a Fighter Squad 1 Paratroopers 1 Bomber Squad 1 Figure 4: An allocation that satis es Commander 2. 6 Figure 3 shows how resource contention arises when a second commander enters the system and wants to conduct a ground attack with one each of a ghter, bomber, and paratrooper squad. Since we cannot satisfy both commanders, we should allocate resources to satisfy the commander desiring a higher value task. Figure 4 shows how we could allocate resources to complete the task of Commander 2. In a more realistic scenario, the air elds are not \used up" when they are allocated, but rather are constrained to provide service to a limited number of squads within a given period of time. We could thus allocate the air elds for each period of time under consideration. The commanders may have varying values for completing their tasks at di erent times. The addition of time-dependent resources and valuations adds a scheduling component to the basic problem of supply chain formation. 2.2 Problem Speci cation We provide a formal description of the problem in terms of bipartite graphs. The two types of nodes represent goods and agents, respectively. A task dependency network is a directed, acyclic graph, (V;E), with vertices V = G [ A: G = the set of goods; A = [ S [ C, the set of agents; = the set of producers; S = the set of suppliers; C = the set of consumers; and a set of edges E connecting agents with goods they can use or produce. There exists an edge hg; ai from g 2 G to a 2 A when agent a can make use of one unit of g, and an edge ha; gi when a can provide one unit of g. If an agent requires two units of a good as input, then we treat each unit as a separate edge hg; aik indexed by k. For instance, if agent a requires two units of g as input, then its input edges are hg; ai1 and hg; ai2. We treat multiple output units in the same manner. The goods can be traded only in integer quantities. The edges can be further characterized by the type of agent involved. A supplier can supply some set of goods, up to some maximum quantity for each, without requiring any input goods: 7 For each supplier s, there there exists a set Gs G, such that for each g 2 Gs there exist edges hs; gik 2 E for one or more k. Each consumer wishes to acquire one unit of one good in a set of some high-level goods: For all c 2 C, there exists a set Gc G, such that hg; ci 2 E for each g 2 Gc. A producer can produce a single unit of an output good conditional on acquiring some input goods: For all 2 : there exists a set I G, such that for each g 2 I there are edges hg; ik 2 E for one or more k, and there exists a single g 2 G n I , such that h ; g i 2 E. A producer's input requirements are complementary in that it must acquire each of its inputs; it cannot accomplish anything with only a partial set. The existence of di erent producers with the same output corresponds to di erent ways that a good can be produced. We consider only acyclic task dependency networks, that is no agent supplies goods that could be used to assemble its inputs. Figure 5 shows an example task dependency network that models the air combat domain problem showed in Figure 3. Here the goods are indicated by circles, the consumers (commanders) and suppliers (air elds) are represented as boxes, and the producers are indicated by curved boxes. An arrow from an agent to a good indicates that the agent can provide that good, and an arrow from a good to an agent indicates that the agent can make us of the good. For instance, the producer labeled \ground attack" requires one unit each of a ghter squad, bomber squad, and paratroopers, in order to provide its output. An allocation is a subgraph 0 = (V 0; E 0) (V;E). For a 2 A \ V 0 and g 2 G \ V 0, an edge ha; gi 2 E 0 means that agent a provides g, and hg; ai 2 E 0 means a acquires g. An allocation contains vertices and edges as follows: 1. An agent is in an allocation graph i it acquires or supplies a good: For a 2 A, we have a 2 V 0 \A i hg; ai 2 E 0 or ha; gi 2 E 0. 8 Airfield 1a Fighter Squad 1 Airfield 1b Bomber Squad 1 Paratroopers 1 Airfield 2a Fighter Squad 2 Airfield 2b Bomber Squad 2 Paratroopers 2 Air Attack Ground Attack Commander 1 Commander 2 Figure 5: A task dependency network for the air combat domain. 2. A good is in an allocation graph i it is bought or sold: For g 2 G, we have g 2 V 0 \G i hg; ai 2 E 0 or ha; gi 2 E 0. A solution is an allocation that forms a partial ordering of feasible production, culminating in consumption. To qualify as a solution, a subgraph must satisfy the following constraints. 1. Some consumer acquires a good it desires: There exists a c 2 C and a hg; ci 2 E such that hg; ci 2 E 0. A solution may involve multiple consumers. If consumer c is in a solution 0 then we say that 0 is a solution for c. 2. All producers are feasible: For all 2 such that h ; g i 2 E 0, if hg; i 2 E then hg; i 2 E 0. Note that the feasibility constraints do not exclude the possibility that a producer acquires some inputs without providing its output. If a producer provides its output, we say that it is active. 3. For each good, supply equals demand: 9 Airfield 1a Fighter Squad 1 Airfield 1b Bomber Squad 1 Paratroopers 1 Airfield 2a Fighter Squad 2 Airfield 2b Bomber Squad 2 Paratroopers 2 Air Attack Ground Attack Commander 1 Commander 2 Figure 6: A solution allocation that satis es Commander 2. For each g 2 G, we have jShg;bi2E0 j = jShs;gi2E0 j. Figure 6 shows a solution allocation for the task dependency network shown in Figure 5. Shaded agents are part of the solution, and dashed arrows indicate input/output relations not part of the solution. Each supplier s has some opportunity cost ocs(fgg) for supplying good g, where hs; gi 2 E. A supplier's total opportunity cost ocs(X) for selling the set of goods X is the sum of its opportunity costs for selling the individual goods. The cost might represent the value s could obtain from putting the goods to some other use, or some actual, direct cost incurred in supplying the goods. We de ne the cost of a solution as the sum of the opportunity costs for supplying the primitive goods1: cost( 0) = X hs;gi2E0 j s2S ocs(fgg): I assume that a consumer has preferences over di erent possible goods, but only wishes to obtain a single unit of one good. Thus, a consumer c obtains value vc(fgg) for obtaining one unit of good g alone, and obtains 1Since the individual supplier cost and the solution cost are both additive, it is irrelevant from a global perspective whether we have a separate supplier for each unit of each good, or a single supplier for all goods. However, we may have cause to break down the suppliers in a particular fashion to conveniently denote a particular problem. 10 value vc(X) = max g2X j hg;ci2E vc(fgg) for obtaining a set of goods X. De nition 1 (value of a solution) The value value( 0) of solution 0 is the di erence between the value obtained by the consumers in the solution, and the cost for supplying the primitive goods: value( 0) = X hg;ci2E0 j c2C vc(fgg) cost( 0) Denote as (;; ;) the allocation in economy where no agent buys or sells any goods, that is, the subgraph of with no vertices or edges. De nition 2 (e cient allocations) De ne V = max(0 ; max value( 0) j 0 is a solution). The set of e cient allocations contains all solutions such that value( ) = V and contains (;; ;) if V = 0. The task dependency network formulation represents a very general class of problems. Here I provide some hints as to its generality. Although a producer only produces a single unit of its output, we can replicate producers to achieve higher levels of production. We can model more complicated consumer value functions through combinations of multiple consumers and producers. For instance, we can represent an additive value function by simply instantiating a consumer for each sum term in the function. If a consumer has preferences over bundles of goods, then we can represent the bundles as output goods of producers, with the inputs of a producer corresponding to the components of the respective output bundle. In this general formulation of the task allocation problem, I do not directly address the problem of scheduling. However, the task dependency network model is su ciently general to include task scheduling in discrete time units, as described in Section 2.3. Task dependency networks can be viewed as AND/OR graphs, with the consumer at the root. Agents correspond to AND nodes, and goods to ORs. One could thus solve task allocation problems in a centralized manner via AND/OR search techniques, with some extra bookkeeping to account for the fact that agents may participate in the solution in at most one way, and that the number of edges leading into a good must be the same as the number 11 leading out in a solution (in other words, to treat these properly as graphs rather than trees). But I assume that we are constrained to solve the problem in a decentralized fashion. Section 3 describes an abstract framework for how a price system can guide decentralized task allocation. Section 4 surveys some candidate protocols for decentralized task allocation, and Section 5 describes and analyzes a particular price-based protocol. Section 6 describes how we can detect quiescence of negotiation in a market protocol in a decentralized manner. 2.3 Scheduling Tasks Often we must contend with time constraints in achieving tasks, in addition to general input/output constraints. In such situations, we must not only determine which agents will complete tasks, but also schedule when the tasks should be accomplished. There are many ways in which we could choose to formulate the scheduling problem, and our choice may depend on the particular problem at hand. In this section I show one way that scheduling can be formulated in the general task dependency network model of task allocation (Section 2.2). Because this formulation of scheduling does not require any additions to the general task allocation model, all analytical properties of the general model apply directly to scheduling problems. I refer to goods that are not speci c to a particular time as atemporal goods. However, we must also consider goods available at particular times. The set of such temporal goods is constructed from the cross product of some underlying set of atemporal goods G , and an integer interval [1; T ], where T is the maximum time under consideration. That is, for every g 2 G and every t 2 [1; T ], there exists a temporal good gt 2 GT , which may be interpreted as \good g provided at time t." Often, it may be necessary to perform a task for longer than one unit of time. In such cases, we consider temporal bundle goods. A temporal bundle good g(x;y) represents the atemporal good g 2 G available at x distinct contiguous times, with the latest time at y, and the earliest time at y x+1. Therefore, the temporal bundle goods are the complete set of goods of the form g(x;y), where 1 y T , and x y T . The asymptotic upper bound on the number of temporal bundle goods is: jGBj = jG j T X y=1 y X x=1 1 12 = jG j T X y=1 y = jG jT (T + 1)=2 = O(jG jT 2): Suppliers may have various supply constraints in a temporal good economy. Some suppliers may o er quantity constrained goods that are \used up" when they supply them. A quantity constrained good pertinent to the air combat domain might be airplane fuel. We can generally model quantity constrained goods as atemporal goods. Other suppliers may be temporally constrained in that they can provide a xed amount of a particular atemporal good g 2 G at any and all time periods t. For instance, in the air combat domain, an air eld isn't \used up" when an airplane takes o , but there is a limit to the number of planes that can use an air eld within a xed period of time. In this case, for supplier s, we would have hs; gti 2 E for all t 2 [1; T ], where gt is the temporal version of g at time t. It is also possible that some suppliers o er temporal goods, but only for subsets of the possible time period [1; T ]. In general we may expect that a consumer would achieve a di erent value for acquiring a good at di erent times. For convenience, we can discard edges hgt; ci for which consumer c would receive zero value for good gt (e.g., time t is too early or too late). The allocations to a producer must correctly re ect the scheduling constraints on both its inputs and its outputs. To be consistent with the temporal model, the time of a producer's output must be no earlier than any of its inputs. Additionally, a producer may take require several time units to produce its output. Let producer start at time i and have production time t. Then each of its input goods are of the form gi or g(t;i+t) and its output is of the form gi+t , for some i. In general, we can assume that if there is some such producer with production time t, then there are O(T t) similar producers, one for each i such that 1 i T t. An arbitrageur is a special kind of producer that converts temporal goods to temporal bundle goods. For each bundle good g(x;y) 2 GB, there is an arbitrageur 2 such that: For some g 2 G , and all t 2 [y x+ 1; y] we have hgt; i 2 E. h ; g(x;y)i 2 E. 13 Airfield 1a Fighter Attack 1 at t=2 Airfield 1b t=1 t=2 t=3 t=4 (2,2) (2,3) (2,4) Fighter Attack 1 at t=3 Fighter Attack 1 at t=4 t=2 t=3 t=4 Fighter Squad 1 Arbitrageurs for Airfield 1a t=1 t=2 t=3 t=4 Arbitrageurs for Airfield 1b Figure 7: A portion of a temporal task dependency network for the air combat domain. 14 Figure 7 show a portion of a temporal task dependency network for the air combat domain (Section 2.1). In this problem, there are four time units during which each air eld is available, and a ghter squad needs two contiguous time slots in an air eld. A ghter squad can use either air eld, but must use the same air eld during both time slots. Thus there are separate arbitrageurs for each air eld. I assume that use of a ghter squadron extends beyond the scheduling horizon, and thus a given ghter squadron cannot be used at multiple time steps. Therefore, I model the ghter squadron as a quantity constrained supplier, and a \ ghter attack 1" producer requires a \ ghter squad 1" good, in addition to a bundle of temporal air eld goods, as input. 3 Price Systems and Competitive Equilibrium In the general equilibrium approach to economic resource allocation, we posit a price system to guide the allocation decisions of agents in a decentralized fashion. A price system p assigns to each good g, a non-negative number p(g) as its price. Intuitively, prices indicate the relative global value of the goods. Therefore, agents may use the prices as a guide to their local decision making. An agent a has quasilinear utility function Ua(X) = va(X) + M for allocation of goods X and \money" M . For consumer c, vc(X) is as de ned in Section 2.2. For supplier s, vc(X) = Phs;gi2X ocs(fgg). For producer , v (X) = 0. I assume that agents wish to maximize their surplus, that is their gain in utility from an allocation. For agent a, good g, and allocation 0 = (V 0; E 0), I de ne the following indicator functions: wS(a; g; 0) = ( 1 if ha; gi 2 E 0 0 otherwise, wB(a; g; 0) = ( 1 if hg; ai 2 E 0 0 otherwise. De nition 3 (agent surplus) The surplus, ( ; 0; p), of an agent in allocation 0 = (V 0; E 0), at some prices p, is the utility gain from an allocation, de ned as follows: 15 (c; 0; p) = vc(fg g) Phg;ci2E0 p(g), for all consumers c 2 C\V 0, and good g such that: g = arg max g j hg;ci2E0 vc(fgg): ( ; 0; p) = wS( ; g ; 0)p(g ) Pg2I wB( ; g; 0)p(g) for all producers 2 \V 0, where g is the output good of and I are the input goods of . (s; 0; p) = Phs;gi2E0(p(g) ocs(fgg)), for all suppliers s 2 S \ V 0. Informally, an allocation 0 is a competitive equilibrium at prices p if 0 balances supply and demand for each good and assigns to each agent a bundle of goods that optimizes the agent's surplus at p. A competitive equilibrium allocation is stable in the sense that no agent would want a di erent allocation at the equilibrium prices. Moreover, a competitive equilibrium allocation is e cient. In this section I discuss properties and existence of a type of approximate equilibria I call -competitive equilibria. I defer to later sections the discussion of how we may arrive at an equilibrium. The and refer to parameters used in the SAMP-SB protocol (Section 5). The SAMP-SB protocol can sometimes arrive at a -competitive equilibrium (Section 5.5). Denote as a(p) the maximum surplus that agent a can obtain at prices p by receiving a non-null allocation. I nd it useful to de ne the function for consumers and producers. For all consumers c 2 C, c(p) = max(0 ; max hg;ci2E[vc(fgg) p(g)]): For all producers 2 , (p) = p(g ) X g2I p(g): De nition 4 ( -competitive equilibrium) An allocation 0 of a task allocation economy is in -competitive equilibrium at prices p i : 1. If c(p) > , then consumer c 2 C buys any one good g, where hg; ci 2 E, such that vc(fgg) p(g) c(p) . If c(p) < 0, then c does not buy a good. If 0 c(p) , then either case may occur, subject to the restriction that vc(fgg) p(g) 0 if c buys g. 16